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DETAILED ACTION 

Remarks 

1 . Claims 1-20 have been examined and rejected. This is the first Office action on 
the merits. 

Drawings 

2. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference character(s) not mentioned in the 
description: Reference characters 134 and 136 in [figure 1]. 

3. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to 
the specification to add the reference character(s) in the description in compliance with 
37 CFR 1 .121(b) are required in reply to the Office action to avoid abandonment of the 
application. Any amended replacement drawing sheet should include all of the figures 
appearing on the immediate prior version of the sheet, even if only one figure is being 
amended. Each drawing sheet submitted after the filing date of an application must be 
labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 
CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. The 
objection to the drawings will not be held in abeyance. 
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Claim Rejections -35USC§112 

4. The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

5. Claim 20 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. 

a. Claim 20 recites the limitation "the system" in [line f/of the claim. There is 
insufficient antecedent basis for this limitation in the claim. 

Claim Rejections - 35 USC § 101 

6. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

7. Claims 1 , 2, 5, and 6 are rejected under 35 U.S.C. 101 because data structures 
not claimed as embodied in computer-readable media are descriptive material per se 
and are not statutory because they are not capable of causing functional change in the 
computer. See, e.g., Warmerdam, 33 F.3d at 1361, 31 USPQ2d at 1760 (claim to a data 
structure per se held nonstatutory). Such claimed data structures do not define any 
structural and functional interrelationships between the data structure and other claimed 
aspects of the invention which permit the data structure's functionality to be realized. 
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8. Claims 19 and 20 are rejected under 35 U.S.C. 101 because claim 19 is not 
limited to tangible embodiments. In view of Applicant's disclosure, specification [page 
12, paragraph 48], the medium is not limited to tangible embodiments, instead being 
defined as including intangible embodiments (e.g., [propagation medium]). As such, the 
claim is not limited to statutory subject matter and is therefore non-statutory. 

To overcome this type of 101 rejection the claims need to be amended to include 
only the physical computer media and not a transmission media or other intangible or 
non-functional media. 

Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

10. Claims 1, 2, 5, 14, 15 are rejected under 35 U.S.C. 102(b) as being anticipated 
by Becker (Pub. No. US 2002/0149617 A1). 

Claims 1, 2, 5 

1 0-1 . Regarding claim 1 , Becker teaches the claim comprising generating a first video 
signal, a first audio signal and a first haptic signal at a first location, generating a second 
video signal, a second audio signal and a second haptic signal at a second location, 
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communicating the first video signal, the first audio signal and the first haptic signal to 
the second location, and communicating the second video signal, the second audio 
signal and the second haptic signal to the first location, by disclosing remote 
collaboration between various persons and groups wherein the collaboration utilizes 
computer-generated information and graphics displays with other high-resolution video 
sources, in a real-time mode [paragraph 2]. As shown in [figure 1], signals are sent from 
a local site to a remote site and vice versa [paragraph 221]. The remote collaboration 
includes the transmission of audio and visual signals [paragraphs 197-205] as well as 
haptic signals [paragraph 86]. 

10-2. Regarding claim 2, Becker teaches the claim wherein communicating to the first 
location is concurrently performed with communicating to the second location 
[paragraph 221]. 

10-3. Regarding claim 5, Becker teaches the claim further comprising the steps of 
integrating the first video signal, the first audio signal, and the first haptic signal into a 
first integrated signal, integrating the second video signal, the second audio signal, and 
the second haptic signal into a second integrated signal, and concurrently 
communicating the first integrated signal to the second location and communicating the 
second integrated signal to the first location, by disclosing that the audio, video, and 
haptic signals may be sent to and from the remote computer using an ATM computer 
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network switch or through Ethernet and TCP/IP [figures 4-D, 4-E, 4-F, 4-G, 5-B/ 
Communication between computers is done in real-time [paragraph 21]. 

Claims 14, 15 

10-4. Regarding claim 14, Becker teaches the claim comprising a first conferencing 
signal originating at a first location, the first conferencing signal comprising an audio 
portion corresponding to sound detected by an audio detection device at the first 
location, by disclosing remote collaboration between various persons and groups 
wherein the collaboration utilizes computer-generated information and graphics displays 
with other high-resolution video sources, in a real-time mode [paragraph 2]. A 
microphone for capturing sounds at a local site is provided [paragraph 203]. 

Becker teaches a video portion corresponding to a video generated by a first 
camera at the first location, by disclosing cameras allow users to see each other as they 
work with the computer imagery [paragraphs 198-200]. 

Becker teaches a haptic portion corresponding to a haptic signal generated by a 
haptic device at the first location, by disclosing sending haptic signals from one location 
to another over an Internet connection [paragraph 86]. 

Becker teaches a second conferencing signal originating at a second location, 
the second conferencing signal comprising a second audio portion corresponding to 
other sounds detected by a second audio detection device at the second location, by 
disclosing a microphone for capturing sounds at the remote site [paragraph 204]. 
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Becker teaches a second video portion corresponding to a second video 
generated by a second camera at the second location, by disclosing that a remote site 
can also allow users to capture a video and present it on display of a local site 
[paragraph 201]. 

Becker teaches a second haptic portion corresponding to. a second haptic signal 
generated by a second haptic device at the second location, by disclosing sending 
haptic signals from one location to another over an Internet connection [paragraph 86]. 

Becker teaches a communication system configured to communicate the first 
conferencing signal to the second location and configured to communicate the second 
conferencing signal to the first location, by disclosing that the audio, video, and haptic 
signals may be sent to and from the remote computer using an ATM computer network 
switch or through Ethernet and TCP/IP [figures 4-D, 4-E, 4-F, 4-G, 5-B]. Communication 
between computers is done in real-time [paragraph 21]. 

10-5. Regarding claim 15, Becker teaches the claim wherein the communication 
system comprises at least one of an internet system, a telephony system, a radio 
frequency (RF) wireless system, a microwave communication system, a fiber optics 
system, an intranet system, a local access network (LAN) system, an Ethernet system, 
a cable system, a radio frequency system, a cellular system, an infrared system, and a 
satellite system, by disclosing that the audio, video, and haptic signals may be sent to 
and from the remote computer using an ATM computer network switch or through 
Ethernet and TCP/IP [figures 4-D, 4-E, 4-F, 4-G, 5-B]. 
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Claim Rejections - 35 USC § 103 

1 1 . The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

12. Claims 3, 4, 6-13, and 16-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Becker (Pub. No. US 2002/0149617 A1) and Shrader (U.S. Patent 
No. 6,639,582 B1). 

Claims 3, 4, 6 

12-1 . Regarding claims 3 and 4, Becker teaches the invention substantially as claimed. 
See section 10-1. Becker further teaches the claim further comprising generating an 
audible sound at the first and second location, the audible sound at the first location 
corresponding to the second audio signal and the audible sound at the second location 
corresponding to the first audio signal and displaying a video at the first and second 
location, the video at the first location corresponding to the second video signal and the 
video at the second location corresponding to the first video signal, by disclosing 
broadcasting audio [paragraph 203] and video [paragraphs 200, 201]. This allows 
collaborators to see and talk with each other as they work with the computer imagery 
[paragraph 198]. 
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Although Becker teaches generating and communicating a haptic signal between 
computers [paragraph 86], Becker does not expressly teach reproducing a haptic image 
at the first and second location, the haptic image at the first location corresponding to 
the second haptic signal and the haptic signal at the second location corresponding to 
the first haptic signal. Shrader teaches an interactive computer controlled system for 
communicating haptic sensory-motor effects between network connected input/output 
devices [column 1, lines 11-14]. Feedback of a resultant or combined haptic sensory- 
motor effect is sent back to a remote terminal and then visa-versa from the remote 
terminal back to a local terminal [column 2, lines 20-36]. This allows human-computer 
interfaces to become more friendly and comfortable to users [column 1 } lines 40-52]. 
Since Becker teaches providing a collaboration environment that enables rich 
interaction to allow natural collaborative behaviors to occur across shared remote 
spaces, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the collaborative environment, sending and receiving 
feedback from haptic sensory-motor effects at two or more networked remote locations 
as taught by Shrader. This would allow for richer interactions among users within the 
collaborative environment. 

12-2. Regarding claim 6, Becker teaches the invention substantially as claimed. See 
section 10-3. Although Becker teaches generating and communicating a haptic signal 
between computers [paragraph 86], Becker does not expressly teach generating an 
integrated haptic signal from the first integrated signal and the second integrated signal, 
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reproducing an integrated haptic image corresponding to the integrated haptic signal at 
the first location, and concurrently reproducing the integrated haptic image at the 
second location. Shrader teaches an interactive computer controlled system for 
communicating haptic sensory-motor effects between network connected input/output 
devices [column 1, lines 11-14], Feedback of a resultant or combined haptic sensory- 
motor effect is sent back to a remote terminal and then visa-versa from the remote 
terminal back to a local terminal [column 2, lines 20-36], The feedback to produce 
concurrent resultant sensory-motor affects at least two remote terminals through the 
combination of the haptic inputs at each terminal [column 4, lines 41-50]. This allows 
human-computer interfaces to become more friendly and comfortable to users [column 
1, lines 40-52], Since Becker teaches providing a collaboration environment that 
enables rich interaction to allow natural collaborative behaviors to occur across shared 
remote spaces, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to include in the collaborative environment, sending and 
receiving feedback from haptic sensory-motor effects at two or more networked remote 
locations as taught by Shrader. This would allow for richer interactions among users 
within the collaborative environment. 

Claims 7-13 

12-3. Regarding claim 7, Becker teaches the claim comprising a video camera at a first 
location configured to capture video and communicate the video to a second location 
and a display at the second location configured to receive and display the 
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communicated video, by disclosing remote collaboration between various persons and 
groups wherein the collaboration utilizes computer-generated information and graphics 
displays with other high-resolution video sources, in a real-time mode [paragraph 2]. 
Cameras allow users to see each other as they work with the computer imagery 
[paragraphs 198-202]. 

Becker teaches an audio input device at the first location configured to capture 
audio and communicate the captured audio to the second location and an audio output 
device at the second location configured to receive and reproduce the communicated 
audio, by disclosing a microphone for capturing sounds at a local site and speakers for 
outputting the captured sounds at a remote site [paragraphs 203-205]. 

Becker teaches a first haptic device at the first location configured to generate a 
haptic signal to communicate the haptic signal to the second location, by disclosing 
sending haptic signals from one location to another over an Internet connection 
[paragraph 86]. 

Becker does not expressly teach a second haptic device at the second location 
configured to receive the haptic signal and produce a haptic image corresponding to the 
communicated haptic signal. Shrader teaches an interactive computer controlled system 
for communicating haptic sensory-motor effects between network connected 
input/output devices [column 1, lines 1 1-14]. Feedback of a resultant or combined haptic 
sensory-motor effect is sent back to a remote terminal and then visa-versa from the 
remote terminal back to a local terminal [column 2, lines 20-36]. This allows human- 
computer interfaces to become more friendly and comfortable to users [column 1, lines 
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40-52]. Since Becker teaches providing a collaboration environment that enables rich 
interaction to allow natural collaborative behaviors to occur across shared remote 
spaces, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to include in the collaborative environment, sending and receiving 
feedback from haptic sensory-motor effects at two or more networked remote locations 
as taught by Shrader. This would allow for richer interactions among users within the 
collaborative environment. 

1 2-4. Regarding claims 8-1 1 , Becker and Shrader teach the claim wherein the first 
haptic device is further configured to detect a force exerted by an object, wherein the 
communicated haptic signal corresponds to the detected object and wherein the second 
haptic device is configured to detect a second force exerted by a second object, 
wherein the communicated haptic signal corresponds to integration of the detected 
forces, by disclosing a touchboard or touchpad of a matrix of rods which are movable 
within a substrate or board by touch via a finger or hand as shown in [Shrader, figures 
3-5]. The finger or hand is used to physically input the haptic forces while the haptic 
forces or effects transmitted or feedback from the other haptic terminal are applied via 
the computer controlling the respective touchboard through an individual small motor 
associated with each rod in the touchboard matrix [Shrader, column 4, lines 12-50]. 

12-5. Regarding claim 12, Becker and Shrader teach the claim further comprising a 
processor configured to integrate the communicated video, audio, and haptic signal into 
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an integrated signal that is communicated to the second location, by disclosing that the 
audio, video, and haptic signals may be sent to and from the remote computer using an 
ATM computer network switch or through Ethernet and TCP/IP [Becker, figures 4-D, 4- 
E, 4-F, 4-G, 5-B]. Communication between computers is done in real-time [Becker, 
paragraph 21]. There inherently must be a processor in order to communicate the signal 
to the second location. 

12-6. Regarding claim 13, Becker and Shrader teach the claim further comprising a 
second video camera at the second location configured to capture a second video and 
communicate the second video to the first location and a second display at the first 
location configured to receive and display the second video, by disclosing that a remote 
site can also allow users to capture a video and present it on display of a local site 
[Becker, paragraph 201]. 

Becker and Shrader teach a second audio input device at the second location 
configured to detect a second audio and communicate the detected second audio to the 
first location and a second audio output device at the first location configured to receive 
and reproduce the communicated second audio, by disclosing a microphone for 
capturing sounds at the remote site and speakers for outputting the captured sounds at 
the local site [Becker, paragraph 204]. 



Claims 16-18 
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12-7. Regarding claim 16, Becker teaches the claim comprising means for 
communicating a first conferencing signal to a first location, the first conferencing signal 
comprising a first video signal, a first audio signal, and a first haptic signal each 
generated at a second location and means for communicating a second conferencing 
signal to the second location, the second conferencing signal comprising a second 
video signal, a second audio signal and a second haptic signal each generated at the 
first location, by disclosing remote collaboration between various persons and groups 
wherein the collaboration utilizes computer-generated information and graphics displays 
with other high-resolution video sources, in a real-time mode [paragraph 2]. As shown in 
[figure 1], signals are sent from a local site to a remote site and vice versa [paragraph 
221]. The remote collaboration includes the transmission of audio and visual signals 
[paragraphs 197-205] as well as haptic signals [paragraph 86]. The audio, video, and 
haptic signals may be sent to and from a remote computer using an ATM computer 
network switch or through Ethernet and TCP/IP [figures 4-D, 4-E, 4-F, 4-G, 5-B]. 
Communication between computers is done in real-time [Becker, paragraph 21]. 

Becker teaches means for displaying the first video signal and the second video 
signal, by disclosing video monitors to display the video signals [paragraphs 200, 201]. 

Becker teaches means for reproducing the first audio signal and the second 
audio signal, by disclosing speakers for reproducing the audio signals [paragraphs 203, 
204]. 

Although Becker teaches generating and communicating a haptic signal 
between computers [paragraph 86], Becker does not expressly teach means for 
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reproducing the first haptic signal and the second haptic signal. Shrader teaches an 
interactive computer controlled system for communicating haptic sensory-motor effects 
between network connected input/output devices [column 1, lines 11-14]. Feedback of a 
resultant or combined haptic sensory-motor effect is sent back to a remote terminal and 
then visa-versa from the remote terminal back to a local terminal [column 2, lines 20- 
36]. This allows human-computer interfaces to become more friendly and comfortable to 
users [column 1, lines 40-52]. Since Becker teaches providing a collaboration 
environment that enables rich interaction to allow natural collaborative behaviors to 
occur across shared remote spaces, it would have been obvious to one of ordinary skill 
in the art at the time the invention was made to include in the collaborative environment, 
sending and receiving feedback from haptic sensory-motor effects at two or more 
networked remote locations as taught by Shrader. This would allow for richer 
interactions among users within the collaborative environment. 

12-8. Regarding claim 17, Becker and Shrader teach the claim further comprising 
means for receiving a second communication signal at the second location, the second 
communication signal comprising a second video signal, a second audio signal, and a 
second haptic signal each generated at the first location, by disclosing that the audio, 
video, and haptic signals may be sent to and from a remote computer using an ATM 
computer network switch or through Ethernet and TCP/IP [Becker, figures 4-D, 4-E, 4-F, 
4-G, 5-B]. Communication between computers is done in real-time [Becker, paragraph 
211 
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Becker and Shrader teach means for displaying the second video signal as a 
second video, by disclosing video monitors to display the video signals [Becker, 
paragraphs 200, 201]. 

Becker and Shrader teach means for reproducing the second audio signal as a 
second audible sound, by disclosing speakers for reproducing the audio signals 
[Becker, paragraphs 203, 204], 

Becker and Shrader teach means for reproducing the second haptic signal as a 
second haptic image, by disclosing that feedback of a resultant or combined haptic 
sensory-motor effect is sent back to a remote terminal and then visa-versa from the 
remote terminal back to a local terminal [Shrader, column 2, lines 20-36]. 

12-9. Regarding claim 18, Becker and Shrader teach the claim further comprising 
means for integrating the first haptic signal and the second haptic signal into an 
integrated haptic signal, means for reproducing an integrated haptic image 
corresponding to the integrated haptic signal at the first location, and means for 
concurrently reproducing the integrated haptic image at the second location, by 
disclosing a touchboard or touchpad of a matrix of rods which are movable within a 
substrate or board by touch via a finger or hand as shown in [Shrader, figures 3-5/. The 
finger or hand is used to physically input the haptic forces while the haptic forces or 
effects transmitted or feedback from the other haptic terminal are applied via the 
computer controlling the respective touchboard through an individual small motor 
associated with each rod in the touchboard matrix [Shrader, column 4, lines 12-40]. The 
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feedback to produce concurrent resultant sensory-motor affects at least two remote 
terminals through the combination of the haptic inputs at each terminal [Shrader, 
column 4, lines 41-50]. 

Claims 19, 20 

12-10. Regarding claim 19, Becker teaches the claim comprising logic configured to 
communicate a first conferencing signal to a first location, the first conferencing signal 
comprising a first video signal, a first audio signal, and a first haptic signal each 
generated at a second location and logic configured to communicate a second 
conferencing signal to the second location, the second conferencing signal comprising a 
second video signal, a second audio signal, and a second haptic signal each generated 
at the first location, by disclosing remote collaboration between various persons and 
groups wherein the collaboration utilizes computer-generated information and graphics 
displays with other high-resolution video sources, in a real-time mode [paragraph 2]. As 
shown in [figure 1] t signals are sent from a local site to a remote site and vice versa 
[paragraph 221]. The remote collaboration includes the transmission of audio and visual 
signals [paragraphs 197-205] as well as haptic signals [paragraph 86]. The audio, video, 
and haptic signals may be sent to and from a remote computer using an ATM computer 
network switch or through Ethernet and TCP/IP [figures 4-D, 4-E, 4-F, 4-G, 5-B]. 
Communication between computers is done in real-time [Becker, paragraph 21]. 

Although Becker teaches generating and communicating a haptic signal 
between computers [paragraph 86], Becker does not expressly teach logic configured to 
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integrate the first haptic signal and the second haptic signal into an integrated haptic 
signal and logic configured to reproduce an integrated haptic image corresponding to 
the integrated haptic signal at the first location and the second location. Shrader 
teaches an interactive computer controlled system for communicating haptic sensory- 
motor effects between network connected input/output devices [column 1, lines 11-14], 
Feedback of a resultant or combined haptic sensory-motor effect is sent back to a 
remote terminal and then visa-versa from the remote terminal back to a local terminal 
[column 2, lines 20-36]. The feedback to produce concurrent resultant sensory-motor 
affects at least two remote terminals through the combination of the haptic inputs at 
each terminal [column 4, lines 41-50]. This allows human-computer interfaces to 
become more friendly and comfortable to users [column 1 } lines 40-52]. Since Becker 
teaches providing a collaboration environment that enables rich interaction to allow 
natural collaborative behaviors to occur across shared remote spaces, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
include in the collaborative environment, sending and receiving feedback from haptic 
sensory-motor effects at two or more networked remote locations as taught by Shrader. 
This would allow for richer interactions among users within the collaborative 
environment. 

12-11. Regarding claim 20, Becker and Shrader teach the claim further comprising logic 
configured to integrate a force detected by a first haptic device that generates the first 
haptic signal into the integrated haptic signal and logic configured to integrate another 
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force detected by a second haptic device that generates the second haptic signal into 
the integrated haptic signal, by disclosing a touchboard or touchpad of a matrix of rods 
which are movable within a substrate or board by touch via a finger or hand as shown in 
[Shrader, figures 3-5]. The finger or hand is used to physically input the haptic forces 
while the haptic forces or effects transmitted or feedback from the other haptic terminal 
are applied via the computer controlling the respective touchboard through an individual 
small motor associated with each rod in the touchboard matrix [Shrader, column 4, lines 
12-50]. 

Conclusion 

13. The prior art made of record on attached form PTO-892 and not relied upon is 
considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R § 
1 1 1 (c) to consider these references fully when responding to this action. The 
documents cited therein teach similar systems for haptic based conferencing. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Alvin H. Tan whose telephone number is 571-272-8595 . 
The examiner can normally be reached on Mon-Fri 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on 571-272-4048. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

AHT 

Assistant Examiner 
Art Unit 2173 




